Document processing

ABSTRACT

Proposed is the use of a document widget for representing a property of a document. The document widget comprises: a human-readable portion for interpretation by a user; and a machine-readable portion representing the document property. By comprising information about a property of a document, a document widget may be processed in accordance with an optical recognition process so as to identify the document widget and enable extraction of the document property.

Benefit is claimed under 35 U.S.C. 119(a)-(d) to Foreign applicationSerial No. 71/CHE/2010 entitled “DOCUMENT PROCESSING” by Hewlett-PackardDevelopment Company, L. P., filed on Jan. 11, 2010, which is hereinincorporated in its entirety by reference for all purposes.

Processing a document to extract information contained in the documentfor subsequent entry and storage in a digital storage system is a commonproblem faced by enterprises. For example, the cost of processing aphysical document is estimated to be ten times the cost of producing thesame document. Problems that are typically encountered relate to thetime and effort involved in extraction of information, entry into adigital system and subsequent verification of the information. Theadditional effort involved in processing the document may also result inadditional costs being incurred.

It is known to encode information in a machine-readable format so thatthe information can be later decoded and extracted by a digital system.Various techniques for encoding information into a machine-readableformat are known and typically comprise encoding information into aone-dimensional (1D) or two-dimensional (2D) pattern. However, suchtechniques are not designed to be human-readable, are not designed fordistributed placement or may require the use of special decodingdevices.

BRIEF DESCRIPTION OF THE EMBODIMENTS

Embodiments are described in more detail and by way of non-limitingexamples with reference to the accompanying drawings, wherein

FIG. 1 is a flow diagram illustrating a method according to anembodiment;

FIG. 2A shows an example of a binary Toeplitz matrix;

FIG. 2B shows a document widget according to an embodiment;

FIG. 3 is a graph of absolute difference of a number of componentsextracted and the number of squares extracted plotted against thresholdvalue to illustrate how a threshold value may be determined forextraction of a document widget;

FIG. 4 illustrates marker sequences used to identify the corners of thedocument widget of FIG. 2B;

FIG. 5 shows a document comprising a plurality of document widgetsaccording to an embodiment;

FIG. 6 shows a printer according to an embodiment; and

FIG. 7 shows a data processing system according to an embodiment.

DETAILED DESCRIPTION OF THE DRAWINGS

It should be understood that the Figures are merely schematic and arenot drawn to scale. It should also be understood that the same referencenumerals are used throughout the Figures to indicate the same or similarparts.

Proposed is the use of what is referred to hereinafter as a “smartwidget” or a “document widget” for representing properties of adocument. By being printed on a document and comprising informationabout the document, a document widget may be processed in accordancewith an optical recognition process so as to identify documentproperties and enable document information to be automaticallyextracted.

Such document widgets comprise a human-readable element or portion thatprovides for a distinction between different document widgets that canbe identified by a person who sees the widgets. The human-readableelement provides human-readability enabling a human reader to identifythe existence of a widget and the type of document property encoded bythe machine-readable portion of the widget.

Embodiments use machine-readable document widgets that can be printed onpaper documents which may help alleviate or reduce the problemsassociated with transferring a paper document into a digital format.

The document widgets may be designed to encode relatively small amountson information (in the order of 10 s of bytes) in a small footprint area(such as less than a square centimeter). By being of a relatively smallsize, the document widgets may be positioned within a document asdesired (for example, adjacent or proximate information of interest).This may be referred to as the ‘distributed placement’ of documentwidgets, which helps to provide context and meaning to the documentwidgets when interpreted by a human. Document widgets may then beselectively processed according to a human user's requirement, avoidingthe drawback of conventional methodologies which require all of themachine readable code provided in a document to be processed in order togain an understanding of what information is encoded.

Further, the document widgets may be adapted to be efficiently extractedfrom scanned documents using digital image processing techniques. Thismay enable extraction of document information using digital imageprocessing techniques that can be embedded in a digital imaging devicesuch as a scanner or digital camera.

Document widgets can also have a digital incarnation so that they can bedynamically modified, created and used as a part of document creationworkflow and finally printed on a paper document. Conversely, a scanneddocument widget can lead to the digital incarnation which is updated andthen finally printed at the end of a workflow.

A document widget may comprise a human-readable portion (such as animage or an alpha-numeric character) and machine readable portioninformation about a document embedded/encoded therein. Thus, embodimentsenable a user to compose a document on paper and then print or stick adocument widget on the paper to create a ‘smart document’. Using animage capture device, an electronic representation of the smart documentmay be created and the electronic representation can then be processedusing image recognition hardware/software to identify the informationabout the document from the document widget. Thus, document informationcan be automatically extracted and stored. The human-readable elementhelps since it may provide a ‘name-space’identifying the machinereadable portion and may allow a user to know the type of contentencoded by the widget even before decoding it. This may be particularlyuseful when there are several widgets and a user only wants toselectively extract certain information from a paper document.

For many people, writing on paper feels more natural than typing and somay be a preferred method for composing a document. Embodiments enableinformation about the document to be combined with, printed onto, orembedded into a paper document, therefore allowing people to compose adocument on paper which can then be processed by a digital system.Information such as the bounding box coordinates of a handwritten fieldto be extracted from the paper document can therefore be provided withthe document.

Embodiments provide a document processing system which enables a personto compose a document on paper including one or more document widgets,capture the document with an image capture device, and process thedocument to extract document information, all without requiringknowledge of at least part of the document information and/or a documentprocessing program. Embedding document information as machine readablecode removes the need to store information regarding the links betweenan image and associated document information.

A system according to an embodiment may comprise the following threemain components: a document widget generation unit for creating adocument widget; an image capture unit for capturing an image of adocument comprising a document widget; and an image processing unit forprocessing the captured image to extract document information from thedocument widget.

For example, the image processing unit may be a suitably arrangeddesktop PC, laptop, or mobile telephony device phone. The image captureunit may be a camera and scanner, and the document widget generationunit may be a printer. Also, it is to be appreciated that anycombination of these components may be integrated within a singledevice. For example, the image capture unit and image processing unitmay be provided by a single mobile telephony device or a laptop having awebcam. Similarly, the document widget generation unit and the imagecapture unit may be provided by a combined printer and scanner device.It will therefore be understood that a conventional image capture devicemay be used in conjunction with hardware and/or software according to anembodiment so as to extend the use of the conventional image capturedevice to document widget generation or processing.

Further, all three components may also be provided in a single devicesuch as a combined printer and scanner device having an integrated imageprocessing unit. Accordingly, embodiments can be provided by astandalone device.

Referring now to FIG. 1, a method according to an embodiment will now bedescribed.

Firstly, in step 100, a user composes a document. The document iscomposed by writing on an item of stationery, such as a piece of paper.Thus, completion of step 100 results in a physical document 105.

The generation may also be part of a batch process and need not beinteractive so that, once the widgets are defined, every document beingproduced by the batch process would also be able to generate thewidgets. In step 110 a document widget representing properties of thedocument is generated. Here, a user specifies the document properties,such as the author, tile and subject matter using a document widgetgeneration unit, which then generates a document widget and prints thedocument widget 115 on an adhesive piece of paper. The document widget115 may also be printed onto a paper document along with the documentcontent, therefore not requiring to be printed on an adhesive piece ofpaper.

Next, in step 120, the document widget 115 is combined with the physicaldocument 105 by sticking the document widget 115 onto the document. Thiscreates a document 125 which can be used by embodiments to automaticallyextract the document information represented by the email stamp 115. Thedocument 125 created by combining the physical document 105 with theemail stamp 115 may therefore be referred to as a smart document 125 fordata processing.

In step 130, an electronic representation of the smart document 125 isgenerated using image capturing means such as a scanner or digitalcamera. Next, in step 140, the electronic representation is processed inaccordance with an optical recognition process so as to identify thedocument widget 115. The recognition process identifies the existence ofthe document widget 115 within the electronic representation of thesmart document 125 and then determines the document information.Determination of the document information is, for example, undertaken byaccessing a data store which stores information regarding associationsbetween document widgets and document information. Where the documentwidget has been generated so as to comprise document information in amachine readable code format, the document information is determined byextracting the document information from the document widget (forexample, by decoding the machine readable code).

Finally, in step 150, the extracted information is processed and storedby a processing unit. Since the document content is provided by theelectronic representation of the smart document 125 and the documentinformation has been identified in step 140, the document contents andinformation can be processed and stored automatically without the userbeing required to specify details of the document using a document/dataprocessing program. Aspects of proposed embodiments will now bedescribed separately as follows.

Document Widget Generation

Different approaches may be used to generate a document widget inaccordance with embodiments. One approach is to generate a documentwidget that provides for human-readability on small footprint area,while at the same time ensuring robustness against printing and scanningdistortions. This is achieved by using a Binary ConstrainedDouble-triangular Toeplitz (BCDT) structure to generate a documentwidget that can store, for instance, 10 bytes of data within an area of0.5 square centimeters (sq. cm.).

A Toeplitz structure is a square matrix in which eachdescending-diagonal from left to right is a constant. One example of abinary Toeplitz matrix is shown as FIG. 2A. A double-triangularstructure may be further used on top of the Toeplitz structure to doublethe storage capacity.

A double-triangular Toeplitz matrix is defined as a square matrix inwhich the upper left and lower right triangular matrices are Toeplitzstructures. Such a Toeplitz structure may be utilized for widgetextraction and decoding. An example of such a BCDT-based email documentwidget is shown in FIG. 2B. This is a magnified version of an emaildocument widget that will be shown in an example below (see FIG. 5).

The main considerations when using a BCDT according to an embodimentare: (a) human-readability; and (b) print-scan inter-symbol interference(ISI). The illustrated embodiment addresses these considerations byusing the central portion of the document widget area for a base iconwhich offers a human-readable visual meaning to be portrayed by thedocument widget. Here, the base icon ‘@’ conveys that the documentwidget contains information relating to one or more email addresses. Theremaining portion of the widget area contains the encoded data in amachine readable format.

To address print-scan ISI, constrained coding techniques with minimumrun length of 3 are used. A maximum run length constraint of 5 may alsobe used so that the widget extraction can be performed using Gaborfilters matched to the allowed run lengths 3, 4, 5 (see the sectionentitled “Document widget Extraction” below). A rate ⅓ code that mapsinput bits to run length-constrained outputs (where B denotes a blackpixel and W denotes a white pixel) can be defined as detailed in Table 1below.

TABLE 1 Previous Output Input BBB WWW BWW WBB WWB BBW 0 BWW BBB WBB BBWBBB WWB 1 WWW WBB WWB BWW BBW WWW

It can be verified that the output sequence has B/W runs only of allowedlengths 3, 4, or 5, for any input bit sequence. Note that the code has amemory of three output symbols, which can be modeled and implemented asa state machine. Also, since the allowed run lengths are only 3, 4, and5, marker sequences of 6 Bs can be inserted to identify the widgetcorners during decoding.

In addition to the constrained code that avoids print-scan ISI,embodiments also use an error correcting code (Reed-Solomon (RS) code)to protect against print-scan errors. The constrained code and RS codework together to provide robustness against printing and scanningdistortions. Furthermore, to improve widget extractor accuracy, thecentral area around the base icon is made circular. This improves theGabor filter response as detailed in the section entitled “DocumentWidget Extraction” below. Finally, a document widget may be encompassedby a 3-pixel-wide quiet zone that helps distinguish the widget from anydocument background.

Document Widget Printing

After document widget generation, the widget can be printed out forfuture use in document creation. For example, a document widget may beprinted onto to adhesive paper so as to provide a sticker which can bestuck onto stationery. Alternatively, a document widget may be printedonto stationery, thereby providing stationery, referred to assmart-paper or smart-stationery, which is adapted to be automaticallyprocessed and provide information encoded by the widget when scanned orphotographed by a device according to an embodiment.

Widget Extraction

A widget extractor module is adapted to provide for detection andextraction of document widgets from scanned documents. The Toeplitzstructure of document widgets according to an embodiment providesspatially localized spectral features which may be utilized in widgetdetection. A Gabor filtering-based approach may be utilized forextracting these spatially localized spectral features.

The traditional Gabor filter is a sinusoidal signal of particularfrequency and orientation that is modulated by a Gaussian envelope. Foraccurate widget extraction, sinusoids of frequencies ⅓, ¼, ⅕, and ⅙ arechosen. The absolute values of the response of the four filters aresummed and normalized to get the final response. As the widgets containblack and white stripes, the Gabor filter response at the widgets issubsequently higher than any response coming from the document texturesor print-scan variations.

One can adaptively threshold the Gabor filter response in order torobustly detect a document widget. At an optimal threshold, the numberof squares detected attains a plateau, and the difference between thenumber of connected components extracted, and the number of squaresdetected attains a minimum. Referring to FIG. 3, this approach entailscalculating, for up to 20 threshold values between the mode and themaximum of the Gabor filter response, a corresponding absolutedifference of the number of components extracted and the number ofsquares extracted attained and these values are plotted on a graph. Theoptimal threshold is identified where the graph attains its firstminima. This helps to choose an appropriate threshold values whichenables the detection and extraction of widgets in the document,irrespective of background textures or print-scan variations.

After choosing the appropriate threshold value, a morphological closingoperation is performed to fill the gaps with a circular structuralelement of six pixels. Here, we note that the region around the baseicon is circular, where there is no Gabor filter response. Since thecircular structure allows maximum surface area for the regions havingGabor filter response, the structuring element of six pixels issufficient to close the gaps and the widgets are extracted as squareconnected components. This parameter can be adaptively tuned to thedocument image resolution.

To reduce the computational complexity of Gabor filtering, the filteringcan be undertaken on a downsized image. The location of the widgets inthe original resolution image is further tuned utilizing the three-pixelwide quite zone in the image. For each of the connected components, thehorizontal and the vertical projection profile is found to locate thequiet zone around the columns and rows, respectively. If the quiet zoneis not found in either the top, right, bottom or left boundaries, thewidget boundary is appropriately adjusted to include the quiet zone.

Document Widget Decoding

After the extraction of the widget from a scanned document, a widgetdecoder can be used to recover the information stored by the widget.

Firstly, the four corners 400 of the widget are detected using themarker sequences of six Bs as shown in FIG. 4. Decoding thresholds for B(black) and W (white) pixel levels are estimated using auto-calibrationfrom the marker sequences. This may take into account the effectsprint-scan processes so as to support working across all printers andscanners.

Since the widget is a BCDT (Binary Constrained Double-triangularToeplitz) structure, it contains some redundancy, and only a few rowsand columns need to be decoded in order to recover all of the encodeddata. This allows for simple, yet robust decoding.

Along each dimension (both rows and columns) of the widget, thefollowing sets of steps are repeated to obtain the encoded data alongthat dimension.

(i) An estimate of the value of the pixel is obtained using bilinearinterpolation. At this step the number of pixels in each dimension isassumed to be known. Each value is classified as either white or blackby comparison against the white and black thresholds obtained fromauto-calibration. At this step, due to the print scan distortions, somevalues may alternatively be classified as being neither black nor whitewhich can later be corrected based on the conditions described in thenext two steps.

(ii) After the initial classification of the values as black or white(or being neither), the sequence obtained is checked for any run lengthviolations i.e. all the run lengths should satisfy the constraint thatthey should be between 3 and 5 as mentioned in the encoder sectionabove. Values that are neither black nor white are classified as whiteor black or kept as neither depending on the run lengths.

(iii) Once the run length constraints are checked, values that are stillclassified as neither black nor white are classified based on thedifference sequence of the values. This is undertaken based on theobservation that there is a unique pattern in the difference sequencefor black to white transition sequence and white to black transition.

(iv) Viterbi decoding is then used to correct errors and resolve theunclassified bits of individual row or column data after step (ii) and(iii). A decoding table for such a Viterbi decoder for the run lengthencoder used is shown in Table 2 below.

TABLE 2 Previous Sequence Current Sequence\Output CurrentSequence\Output BBB BWW\0 WWW\1 WWW BBB\0 WBB\1 BWW WBB\0 WWB\1 WBBBBW\0 BWW\1 WWB BBB\0 BBW\1 BBW WWB\0 WWW\1

(v). Data from the two rows and columns are then consolidated to obtainthe encoded data, to be decoded by the RS decoder for extracting thedata stored in the widget.

Referring now to FIG. 5, examples of document widgets and their use willnow be described.

A document 500 comprises first 501 to eighth 508 document widgetspositioned at various locations in the document 500. Each documentwidget comprises a human-readable portion and a computer-readableportion. Here, the human-readable portion comprises a base icon situatedat the centre of the document widget which is adapted to indicate apurpose/use of the document widget to a human user/reader of thedocument. The computer-readable portion comprises machine-readable codesurrounding the human-readable portion (i.e. the base icon) and containsencoded information about the document. The specific documentinformation encoded by the machine-readable code depends on thepurpose/use of the document widget (as will now be shown throughillustration of the different document widgets in the document 500 ofFIG. 5).

First 501 and fourth 504 document widgets in the document 500 of FIG. 5are date widgets and each positioned adjacent to respective dateinformation fields in the document 500. Each date widget 501 and 504 hasa base icon comprising the letter “d” to indicate that the widget is adate widget for representing date information. Each date widget 501 and504 also comprises machine-readable code representing date informationwhich can be checked against date information provided in the respectiveadjacent date field. For example, the machine-readable code of the firstwidget 501 comprises encoded information representing the creation dateof the document 500. This should match the date provided in the datefield adjacent the first widget 501 and so can be used to check againstbusiness logic when the document is scanned.

The second document widget 502 in the document 500 is an invoice numberwidget 502 which is positioned adjacent to a respective invoice numberfield 510 in the document 500. The invoice number widget 502 has a baseicon comprising the symbols “i#” to indicate that the widget is aninvoice number widget for representing an invoice number. The invoicenumber widget 502 also comprises machine-readable code representing theinvoice number which can be checked against invoice number provided inthe adjacent invoice number field. For example, the machine-readablecode of the invoice number widget 502 comprises encoded informationrepresenting an automatically created invoice number for the document500 according to an invoice number database. This should match theinvoice number provided in the invoice number field adjacent the secondwidget 502 and therefore can be used to check against business logicwhen the document is scanned.

The third document widget 503 in the document 500 is a name widget 503which is positioned adjacent to a respective name field in the document500. The name widget 503 has a base icon comprising the letter “n” toindicate that the widget is a name widget for representing nameinformation. The name widget 503 also comprises machine-readable coderepresenting a name which can be checked against name informationprovided in the adjacent name field. For example, the machine-readablecode of the name widget 503 comprises encoded information representing aname according to a client database. This should match the nameinformation provided in the name field adjacent the third widget 503 andtherefore can also be used to check against business logic when thedocument is scanned.

Fifth 505 and sixth 506 document widgets in the document 500 of FIG. 5are signature widgets and each positioned adjacent to first 511 andsecond 512 signature fields, respectively, in the document 500. Eachsignature widget 505 and 506 has a base icon comprising the letter “s”to indicate that the widget is a signature widget for representingsignature information. The signature widgets 505 and 506 also comprisemachine-readable code representing the relative coordinates of the first511 and second 512 signature fields, respectively. For example, themachine-readable code of the fifth widget 505 comprises encodedinformation representing the relative location of the first signaturefield 511. The fifth widget 505 can therefore be used to verify whetherthe document has been signed at the location of the first signaturefield 511, thereby enabling real-time verification of the presence of asignature at a predetermined location in the document 500. Similarly,the machine-readable code of the sixth widget 506 comprises encodedinformation representing the relative location of the second signaturefield 512 and can therefore be used to verify whether the document hasbeen signed at the location of the second signature field 512 when thedocument 500 is scanned and processed according to an embodiment.

The seventh document widget 507 in the document 500 is a web addresswidget 507 which is positioned adjacent to a respective web address (orURL) in the document 500. The web address widget 507 has a base iconcomprising the letter “w” to indicate that the widget is a web addresswidget for representing a URL of a web page. The web address widget 507also comprises machine-readable code representing the web address whichcan be used to automatically generate a link to the web address of theweb page. For example, the machine-readable code of the web addresswidget 507 comprises encoded information representing the web address ofthe web page which provides further information regarding term andconditions applicable to the document 500. When the document is scanned,the web address widget 507 can be used to generate a hyperlink to theweb address specified by the web address widget 507 which enables a userto simply click on the hyperlink in order to navigate to the web pagewithout being needed to manually open a web browser program and type theweb address into an address bar of the web browser program.

The eighth document widget 508 in the document 500 is an email addresswidget 508 which is positioned adjacent to a respective email addressfield 513 in the document 500. The email address widget 508 has a baseicon comprising the icon “@” to indicate that the widget is an emailaddress widget for representing an email address. The email addresswidget 508 also comprises machine-readable code representing therelative coordinates of the email address field 513 which can be used toautomatically generate a link to the email address. For example, whenthe document is scanned, the email address widget 508 can be used togenerate a hyperlink to the email address specified by the email addressfield 513. The generated hyperlink then enables a user to simply clickon the hyperlink in order to compose an email address to the emailaddress specified by the email address field 513 without requiring theuser to manually open an email program and type the email address intoan address bar of the email program.

After scanning, the document widgets may also have a digitalrepresentation, for example, as another layer of the document inaddition to the image of the document. This would enable software thatprocesses digital images to also process the paper widgets, and be ableto modify them suitably, without processing/modifying the image of thedocument.

The following examples illustrate the use of particular embodiments:

Example 1

A customer goes to an enterprise website and enters online informationin a pre-structured form (for example an income tax return form). As apart of the process, the customer is expected to print the form, signit, and attach supporting documents and send the physical documents tothe enterprise for completing the transaction. These forms when printedwith widgets allow the scanning workflow of the enterprise toautomatically extract fields of interest and compare them with theonline submitted information prior to further processing.

Example 2

An enterprise prints forms that need to be filled-in and submitted bythe customer (for example loan application forms). These are printedwith widgets so that certain fields on the form such as the signaturefields, date fields, etc can be automatically identified and processedas a part of a document processing workflow once the forms aresubmitted. The processing could include checking if signatures have beenmade, extracting handwriting and carrying out recognition processes onthe extracted handwriting.

Example 3

An enterprise receives several documents from its vendors (such asinvoices). These need to be processed by an accounts payable workflow ofthe enterprise. The invoices have document widgets printed on them(based on templates or tools provided by the enterprise to the vendor).This enables parts of the workflow in the enterprise to be automatedwhere manual transcription would otherwise typically be required.

Example 4

A common problem faced by financial institutions is real-timeverification of the presence of customer signatures on all pages ofsubmitted forms. Conventionally, this is undertaken manually or usingbackend software which is not real-time. Document widgets according toan embodiment can be used to perform real-time signature detectionthrough identification of signature fields where one or more signaturesare expected, thus saving re-routing overhead.

Example 5

Another potential application is with handheld All-in-One (AiO) printersand mail packages. Each mail package, depending on the manufacturingprocess, may have a code that ties it back to a location, date, batch,and machine that it was manufactured by. Document widgets could be usedin the packaging process using the AiO printer to label the box withthat same data as the mail package is filed. As a document widget canstore data in a smaller area than a traditional barcode, it enables theAiO to potentially put more than one widget on a tag to be scanned at alater date. This unique data then makes this particular box traceablethrough the entire process. The document widget may also be used in theshipping documents associated to that mail package.

Example 6

A document can be created which contains the lyrics of a song. A firstdocument widget (an author widget) can be provided in the document whichhas a human-readable portion comprising the letter “A” and amachine-readable portion comprising encoded information about the authorof the lyrics. Also, a second document widget (a music widget) can beprovided in the document which has a human-readable portion comprising asymbol for a musical note (for example, a

) and a machine-readable portion comprising encoded information aboutthe music composition for the lyrics. Thus, a user can scan the documentand be provided access to the music composition or other lyrics composedby the author.

Other potential applications of widgets include: interactive books,product and customer support information on packaging labels, personalinformation on boarding passes/tickets/business cards, documenttrack-and-trace using unique ID, URL information and multimedia links inweb-printing, applications like simple internet prints (SIPs) so thatpaper documents stay ‘connected’ to the web even after printing,automated re-entry of data from filled forms, secure information onconfidential documents, and other intelligent processing applicationsfor paper documents.

It should also be appreciated that a conventional camera or scanner maybe used to capture an image of a document for transmission. The capturedimage can then be provided to a computer or other processing devicewhich is adapted to detect and decode a document widget present in theimage and extract/use document information represented by the documentwidget.

Referring now to FIG. 6, a printer 600 according to an embodiment willnow be described. The printer 600 comprises a user interface 610 adaptedto receive a user input defining document properties such as details ofthe author, a summary of its contents, and/or location(s) of one or moredata fields in the document. Based on the user input, document widgetgeneration means (not visible) integrated within printer are adapted togenerate one or more document widgets representing the user input(s).The printer is adapted to print the document widget(s) using theconventional printing means 615 of the printer. Such a document widgetis adapted to be processed in accordance with an optical recognitionprocess so as to identify the document information represented by thedocument widget.

It will be understood that the printer of FIG. 6 can therefore provide apre-prepared item of stationery comprising a document widgetrepresenting information regarding a document to be made using the itemof stationery. The document widget is adapted to be processed inaccordance with an optical recognition process so as to identifydocument information and instruct a processing unit regarding details ofthe document. The pre-addressed item of stationery may comprise asignature or message portion, such a blank area, within whichinformation is to be provided by a user. The location of the signatureor message portion may then be encoded by the machine readable codeportion of the document widget, thereby enabling the automaticidentification and extraction of information entered into the signatureor message portion by a user.

Referring back to FIG. 6, the printer 600 also comprises a documentscanning unit 620 adapted to scan a document placed thereon so as togenerate an electronic representation of the document. The electronicrepresentation of the document is provided to a document widgetprocessing unit 630 integrated within the printer 600 and adapted todetect and decode a document widget in accordance with an embodiment.

The document widget processing unit 630 identifies the existence of adocument widget within an electronic representation provided to it andthen determines document information encoded by a machine readableportion of a detected document widget. Determination of the documentinformation is, for example, undertaken as described above in thesection entitled “Document Widget Decoding”.

The document widget processing unit 630 is also connected to acommunication unit 640 integrated within the printer 600. Thecommunication unit 640 is adapted to provide access to the internet viaa wireless communication link 645, wherein an electronic representationof a document scanned by the printer can be communicated to anothercomputing device via the internet. The communication unit 640 may accessan internet address, email address or URL provided to it from thedocument widget processing unit 630. Alternatively, a user of theprinter may be prompted, via the user interface, to confirm whether ornot an address identified by the document widget processing unit 630 isto be used or accessed.

In this regard, the user may use the user interface to modify, add,delete or correct document information represented by a document widget.A user may therefore correct or update an address, location or otherdocument-related information identified by a document widget, andinformation about the new or updated information may be stored withinthe printer so that future instances of the same document widget stampresult in the correct/updated information being identified by thedocument widget processing unit 630.

Thus, it will be understood that the printer 600 can be used with smartpaper or stationery comprising a document widget so as to identifydocument information and to extract the document information from thedocument widget.

Turning now to FIG. 7, a data processing system 700 in accordance withan embodiment is shown. A computer 710 has a processor (not shown) and acontrol terminal 720 such as a mouse and/or a keyboard, and has accessto an electronic database stored on a collection 740 of one or morestorage devices, e.g. hard-disks or other suitable storage devices, andhas access to a further data storage device 750, e.g. a RAM or ROMmemory, a hard-disk, and so on, which comprises the computer programproduct implementing at least part of a method according to anembodiment. The processor of the computer 710 is suitable to execute thecomputer program product implementing a method in accordance with anembodiment. The computer 710 may access the collection 740 of one ormore storage devices and/or the further data storage device 750 in anysuitable manner, e.g. through a network 730, which may be an intranet,the Internet, a peer-to-peer network or any other suitable network. Inan embodiment, the further data storage device 750 is integrated in thecomputer 710.

Experimental Evaluation

Extensive testing of document widgets has been carried out over a testset comprising business forms, invoices, and purchase orders. As a testof robustness, three different document background shades wereused—white, light, and dark backgrounds. In total, two hundred documentwidgets on thirty five different documents with different backgroundshades were encoded, extracted, and decoded.

No extraction failures (false negatives or false positives) wereencountered, which suggests that widget extraction according to anembodiment is robust. Further, widget decoding was also shown to berobust since the percentage of data bytes being incorrectly decoded wasfound to less than 1%.

The experiments demonstrated that document widgets could be easily bepositioned within a document, whereas conventional machine readablecodes were of a larger size meaning that they could only be placed in arestricted number of document locations (such as in isolated spaces ordocument margins).

It will be appreciated that embodiments provide advantages which can besummarized as follows:

Document widgets according to embodiments comprise human-readablecontent (for example, text or icons) which makes it possible for a humanuser to interpret the function of the widget. For instance, thehuman-readable portion of an email widget may be symbol “@”. Thisprovides context and meaning to a human reader, and makes the documentwidget amenable for usage with hand-held scanning devices where one candecide the machine-readable information they desire to extract from adocument by looking at the human-readable elements on the widget.

Document widget according to embodiments may be adapted to be easy todetect and extract from a printed document, and hence the extractionprocess can be embedded within a scanning devices or scanning software.Document widgets may be small in size (when compared to conventionalmachine readable codes such as barcodes) so that they can be placedanywhere on the paper document and in close proximity to relativecontents of interest. This may be referred to as distributed placement.Distributed placement can also lead to higher overall reliability sinceeven if one document widget cannot be decoded, the rest of theinformation on the document can still be determined.

Document widgets according to an embodiment may have adequate datacapacity so that they can capture machine readable information thatwould be useful while processing the paper document. For instance, theinformation could be the relative coordinates of a data field from whichdata/information is to be extracted from a document. Since documentwidgets may comprise a human-readable portion, a namespace around themcan be defined so that different sets of widgets can be used asappropriate for different applications. Based on the type of widget, theway information is encoded can be determined.

It should be noted that the above-mentioned embodiments areillustrative, and that those skilled in the art will be able to designmany alternative embodiments without departing from the scope of theappended claims. In the claims, any reference signs placed betweenparentheses shall not be construed as limiting the claim. The word“comprising” does not exclude the presence of elements or steps otherthan those listed in a claim. The word “a” or “an” preceding an elementdoes not exclude the presence of a plurality of such elements.Embodiments can be implemented by means of hardware comprising severaldistinct elements. In the device claim enumerating several means,several of these means can be embodied by one and the same item ofhardware. The mere fact that certain measures are recited in mutuallydifferent dependent claims does not indicate that a combination of thesemeasures cannot be used to advantage.

1. A method for preparing a document to be processed, the methodcomprising; specifying a property of the document; and generating adocument widget representing the specified document property, thedocument widget comprising: a human-readable portion for interpretationby a user; and a machine-readable portion representing the specifieddocument property, wherein the document widget is adapted to beprocessed in accordance with an optical recognition process so as toidentify the document widget and enable extraction of the specifieddocument property.
 2. A method of processing a document, the methodcomprising the steps of: processing an electronic representation of thedocument in accordance with an optical recognition process so as todetect the presence of a document widget in the electronicrepresentation of the document, the document widget comprising: ahuman-readable portion for interpretation by a user; and amachine-readable portion representing a document property; andprocessing the detected document widget to extract the document propertyrepresented by the machine-readable portion of the document widget.
 3. Amethod of processing a document, comprising: providing a document;preparing the document according to claim 1; combining the document andthe document widget so as to create a document for transmission;processing the document according to claim
 2. 4. The method of claim 1,wherein the human readable portion of the document widget comprised atleast one of: an image, an icon and an alphanumeric character.
 5. Themethod of claim 1, wherein the document property comprises at least oneof: an author of content of the document; the location of informationwithin the document; information content of the document; and a creationdate of the document.
 6. Apparatus for preparing a document to beprocessed, the method comprising; a user interface adapted to specify aproperty of the document; and a document widget generating unit adaptedto generate a document widget representing the specified documentproperty, the document widget comprising: a human-readable portion forinterpretation by a user; and a machine-readable portion representingthe specified document property, wherein the document widget is adaptedto be processed in accordance with an optical recognition process so asto identify the document widget and enable extraction of the specifieddocument property.
 7. Apparatus for processing a document, the methodcomprising the steps of: a document widget detection unit adapted toprocess an electronic representation of the document in accordance withan optical recognition process so as to detect the presence of adocument widget in the electronic representation of the document, thedocument widget comprising: a human-readable portion for interpretationby a user; and a machine-readable portion representing a documentproperty; and a document widget decoding unit adapted to extract thedocument property represented by the machine-readable portion of thedocument widget.
 8. An image capture device comprising: image capturingmeans adapted to generate an electronic representation of a document;and apparatus for processing a document according to claim
 7. 9. Aprinter comprising: apparatus for processing a document according toclaim 7; and printing means adapted to print the document widget.
 10. Apre-prepared document comprising a document widget representing aproperty of the document, the document widget comprising: ahuman-readable portion for interpretation by a user; and amachine-readable portion representing the document property, and whereinthe document widget is adapted to be processed in accordance with anoptical recognition process so as to identify the document widget andenable extraction of the document property.
 11. A pre-prepared documentaccording to claim 10, further comprising an Information portion withinwhich information is to be provided by a user of the document, andwherein the document property comprises the location of informationportion within the document.
 12. A computer program product arranged to,when executed on a computer, cause the computer to execute the steps of:generating a document widget representing a specified document property,the document widget comprising: a human-readable portion forinterpretation by a user; and a machine-readable portion representingthe specified document property, wherein the document widget is adaptedto be processed in accordance with an optical recognition process so asto identify the document widget and enable extraction of the specifieddocument property.
 13. A computer program product arranged to, whenexecuted on a computer, cause the computer to execute the steps of:processing an electronic representation of a document in accordance withan optical recognition process so as to detect the presence of adocument widget in the electronic representation of the document, thedocument widget comprising: a human-readable portion for interpretationby a user; and a machine-readable portion representing a documentproperty; and processing the detected document widget to extract thedocument property represented by the machine-readable portion of thedocument widget.
 14. A computer-readable data storage medium comprisingthe computer program product of claim
 13. 15. A data processing systemcomprising: data storage means for storing information; a computerprogram memory comprising the computer program product of claim 13; anda data processor having access to the computer program memory and thedata storage means, the data processor being arranged to execute saidcomputer program product.